Pattern recognition has been accomplished in various ways in the prior art. One of the best known methods of pattern recognition is typified by a simple radar system wherein a beam of electromagnetic energy illuminates a target and is backscattered to a receiver set which is coupled to a computer that analyzes the back-scattered signal and forms an image of the target. Similarly, sonar systems accomplish the same result with acoustical type signals.
Regardless of the transmission and receiving apparatus used in these systems, a multi-purpose, digital computer is continually utilized to perform complex calculations to obtain an output which identifies the input signal. The types of computers used in the prior art to perform such calculations have been exclusively sequential machines that require sophisticated programming to effectively perform pattern recognition algorithms such as Fourier transforms, fast Fourier transforms and similar types of algorithms known to those with ordinary skill in the art.
A major drawback which exists with the use of digital, sequential computers in pattern recognition systems is the inherent limitation of these computers to perform their function only in a strictly sequential fashion. It is known that sequential, digital computers perform one step of a process or algorithm over each machine cycle. In this manner, successive iterations are repeated over a large number of computer machine cycles of a complex algorithm in order to perform pattern recognition and other computer functions.
Depending upon the complexity of the algorithm, the digital computer must perform enormous numbers of machine cycles to form the complete solution of a complex algorithm. For example, when higher order differential equations must be solved simultaneously or when a large number of differential equations must be solved either simultaneously or sequentially, the number of machine cycles required to solve the equations increases drastically. With these drastic increases in machine cycles comes an increased time period for the digital, sequential computer to perform a complete analysis of incoming data. Those skilled in the art will appreciate that complete and useful pattern recognition with such digital computers can take hours or even days. Thus, the use of digital computers generally does not allow pattern recognition in "real-time."
There is therefore a long-felt need in the computer art for a machine which can drastically reduce the time required to perform algorithmic tasks and to provide methods and systems for fast and efficient pattern recognition. Some form of parallel processing of incoming signals could perform this function, also, the use of a parallel processor or a machine capable of inherent parallelism could allow pattern recognition of a complex signal in real-time.
An additional problem which has existed in the computer and pattern recognition arts arises from the requirement that signals be resolved into digital components before they may be processed by a sequential, digital computer. This requires that all incoming signals be first "digitized" by an "analog to digital" component of the pattern recognition system before the digital computer can begin processing the signal with its particular pattern recognition algorithm. This places many burdens on prior art pattern recognition systems in that it requires expensive hardware to implement analog to digital conversion and increases the overall processing time of such systems by requiring the analog to digital conversion step. Thus, a pattern recognition system which utilizes incoming analog signals directly without analog to digital conversion is highly desirable. Such a system has not been known heretofore in the art, however.
Additionally, it is highly desirable to utilize systems for pattern recognition that employ parallel processing of analog signals. Such systems also have not been known in the pattern recognition art. Thus, there is a continuing need for a computer system which utilizes analog signals and performs parallel processing. This need requires an effective system to achieve fast, parallel processing of analog signals.
Apparatus have been developed which simulate or approximate certain aspects of the behavior of neural networks. An example of such a system is embodied in U.S. Pat. No. 4,773,024 to Faggin et al., which discloses a recognize-only embodiment of a recognition matrix having contacts comprised of a forward matrix and a reverse matrix. The contacts disclosed in Faggin et al. are permanently programmed by the user for a class of events and are therefore static. The user typically performs a learning function on a computer for all the events which the system will be programmed to recognize. The pattern of convergence responses and contact structure characteristics which cause convergence responses for the class of events as a whole are then examined and optimized for maximum recognition power and minimal confusion. This pattern of convergence responses is permanently programmed in the contacts of the program reverse matrices.
A similar system is disclosed in U.S. Pat. No. 4,774,667 to Buckley, wherein self-organizing circuits connected to receive a plurality of input signals representing constituent elements of input information are taught. The self-organizing circuits disclosed in Buckley are operable to effect identification of the pattern of constituent elements by combining the influence that each constituent element has on the pattern of constituent elements. A mechanism is provided to modify the influence which each constituent element has on the signal pattern of constituent elements based upon cumulative Boolean functions between the input signals to each circuit output. Furthermore, a mechanism connected to vary the influences based upon competition among the input signals is provided by Buckley in col. 6, line 9 through col. 9, line 2 thereof.
In addition, electronic circuits which mimic neural networks and associative memories are taught in U.S. Pat. No. 4,660,166 to Hopfield, wherein the use of amplifiers connected to a matrix of input and output conductors to produce stored outputs in response to input signals is disclosed. Each connection is implemented with a network of resistors connected to the inputs and outputs in the amplifiers. The resistive values are selected to satisfy the circuit's "equation of motion." The network disclosed in the Hopfield patent is driven to a stable state at the complementary output of the amplifiers which provide an output code word that approximates the problem's solution as described in Hopfield, col. 6, line 10 through col. 10, line 7 thereof.
The aforementioned patents do not solve a long-felt need in the art for methods and apparatus which can drastically reduce the time required to achieve analog processing of data and pattern recognition. While the aforementioned patents provide a modicum of parallel processing, they generally either rely partially on standard digital computers for their data processing capabilities or do not themselves provide pattern recognition but merely pre-process analog data for ultimate conversion to digital signals and subsequent digital processing.
One of the inventors of the subject matter herein claimed and disclosed published a paper which theoretically defined neuron behavior and modelled artificial neurons comprising electronic components based on the input-output relationships of real brain neurons. See. P. Mueller, T. Martin and F. Putzrath, "General Principals of Operations in Neuron Nets with Application to Acoustical Pattern Recognition," reprinted in Biological Prototypes and Synthetic Systems, Vol. 1, p. 192-212 (1962). In the aforementioned paper, a neuron's behavior as a logic device was disclosed. As described therein, the neuron has both excitatory and inhibitory inputs and excitatory and inhibitory feedback which cause the neuron to fire when the combination of these inputs exceeds a threshold voltage. Because firing neurons can be observed without outputting a uniform voltage, Boolean algebra was disclosed to be useful as a tool for quantitative treatment of the relationship between the input and the output of the neuron. Additionally, examples of electronic neurons which simulate the behavior of brain neurons were described in this paper.
As noted in the above-mentioned paper, general electronic neurons may approximate the basic properties of biological neurons. One of inventors of the subject matter herein claimed and disclosed has also recognized that artificial neurons so constructed would be particularly useful in artificial neural networks for speech recognition. See P. Mueller, "Principles of Temporal Pattern Recognition in Artificial Neuron Nets with Application in Speech Recognition," reprinted in Artificial Intelligence. IEEE, pp. 138-44 (1963).
While the general properties of artificial neurons modelled after brain neurons has thus been known in the art at least since the early 1960s, there remains a long-felt need in the art for artificial neural networks which function as general purpose neural computers. None of the aforementioned patents or papers disclose systems which solve a long-felt need in the art for electronic artificial neural networks that process data in analog form, thereby providing pattern recognition and general purpose neural computing.